A Matlab Toolbox for Music Information Retrieval

We present MIRToolbox, an integrated set of functions written in Matlab, dedicated to the extraction from audio files of musical features related, among others, to timbre, tonality, rhythm or form. The objective is to offer a state of the art of computational approaches in the area of Music Information Retrieval (MIR). The design is based on a modular framework: the different algorithms are decomposed into stages, formalized using a minimal set of elementary mechanisms, and integrating different variants proposed by alternative approaches – including new strategies we have developed –, that users can select and parametrize. These functions can adapt to a large area of objects as input. This paper offers an overview of the set of features that can be extracted with MIRToolbox, illustrated with the description of three particular musical features. The toolbox also includes functions for statistical analysis, segmentation and clustering. One of our main motivations for the development of the toolbox is to facilitate investigation of the relation between musical features and music-induced emotion. Preliminary results show that the variance in emotion ratings can be explained by a small set of acoustic features. 1 Motivation and approach MIRToolbox is a Matlab toolbox dedicated to the extraction of musicallyrelated features in audio recordings. It has been designed in particular with the objective of enabling the computation of a large range of features from databases of audio files, that can be applied to statistical analyses. We chose to base the design of the toolbox on Matlab computing environment, as it offers good visualisation capabilities and gives access to a large variety of other toolboxes. In particular, the MIRToolbox makes use of functions available in public-domain toolboxes such as the Auditory Toolbox (Slaney, 1998), NetLab (Nabney, 2002), or SOMtoolbox (Vesanto, 1999). Other toolboxes, such as the Statistics toolbox or the Neural Network toolbox from MathWorks, can be directly used for further analyses of the features ex2 Olivier Lartillot, Petri Toiviainen, and Tuomas Eerola tracted by MIRToolbox without having to export the data from one software to another. It appeared that such computational framework, because of its general objectives, could be useful to the research community in Music Information Retrieval (MIR), but also for teaching. For that reason, a particular attention has been paid concerning the ease of use of the toolbox. The functions are called using a simple and adaptive syntax. More expert user can specify a large range of options and parameters. The different musical features extracted from the audio files are highly interdependent: in particular, as can be seen in figure 1, some features are based on same initial computations. In order to improve the computational efficiency, it is important to avoid redundant computations of these common components. Each of these intermediary components, and the final musical features, are therefore considered as building blocks that can been freely articulated one with each other. Besides, in keeping with the objective of optimal ease of use of the toolbox, each building block has been conceived in a way that it can adapt to the type of input data. Hence, in figure 1, the computation of the MFCCs can be based on the waveform of the initial audio signal, or on the intermediary representations such as spectrum, or mel-scale spectrum. Similarly, the autocorrelation method will behave differently with audio signal or envelope, and can adapt to frame decompositions.This decomposition of the all set of feature extraction algorithms into a common set of building blocks has the advantage of offering a synthetic overview. 2 Feature extraction Figure 1 shows an overview of the main features considered in the toolbox. All the different processes start from the audio signal (on the left) and form a chain of operations developed horizontally rightwise. The vertical disposition of the processes indicates an increasing order of complexity of the operations, from simplest computation (top) to more detailed auditory modelling (bottom). Each musical feature is related to the different broad musical dimensions traditionally defined in music theory. In bold are highlighted features related to pitch, to tonality (chromagram, key strength and key Self-Organising Map, or SOM) and to dynamics (Root Mean Square, or RMS, energy). In bold italics are indicated features related to rhythm: namely tempo, pulse clarity and fluctuation. In simple italics are highlighted a large set of features that can be associated to timbre. Among them, all the operators in grey italics can be in fact applied to many others different representations: for instance, statistical moments such as centroid, kurtosis, etc., can be applied to either spectra, envelopes, but also to any histogram based on any given feature. One of the simplest features, zero-crossing rate, is based on a simple description of the audio waveform itself: it counts the number of sign changes of the waveform. Signal energy is computed using root mean square, or RMS A Matlab Toolbox for Music Information Retrieval 3 Audio signal waveform Zero-crossing rate RMS energy Envelope Attack/Sustain/Release Envelope Autocorrelation Tempo Key strength Key SOM